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Executive Summary 


This report explores the early implementation of state-differentiated recognition, accountability and 
support systems under the U.S. Department of Education’s Elementary and Secondary Education Act 
(ESEA) flexibility initiative, announced in September 2011. The findings are based on telephone 
interviews conducted with a sample of state, district and school-level officials in late 2013 and early 
2014, and a review of relevant policy documents that were available on state and local education agency 
websites during this same time period. Participants in the state and district sample included officials 
from 12 state education agencies with approved ESEA flexibility requests and 22 districts located within 
these 12 states. The school-level sample drew from the state and district sample and participants 
included principals of 25 Title | schools located within a subset of 12 districts and six states. Although the 
state, district and school samples were diverse, the states, districts and schools were not randomly 
selected and are not representative of the larger population of 35 states whose ESEA flexibility requests 
had been approved at the time of this study and the districts and schools within these states. Thus, the 
findings presented in this report are not generalizable. 


States that receive ESEA flexibility are required to implement a state-based accountability system for all 
local education agencies (LEAs) in the state and for all Title | schools in these LEAs. These systems must 
provide differentiated recognition, accountability and support to schools based on their performance— 
and change in performance over time—with respect to academic achievement and graduation rates of 
all students and subgroups of students. State systems must identify and support at least three types of 
Title | schools: priority schools, focus schools and reward schools.’ States also are directed to provide 
incentives and support to encourage continuous improvement in other Title | schools that are not 
making progress in improving student achievement and narrowing achievement gaps (U.S. Department 
of Education, 2012a; U.S. Department of Education, 2012b). These categories of Title | schools are 
identified as follows: 


e Priority school: A Title | school identified as among the lowest performing schools in the state 
and, at the high school level, with a graduation rate less than 60 percent over a number of years. 
A priority school may also be a Tier | or Tier II school under the School Improvement Grant (SIG) 
program that is using SIG funds to implement a school intervention model. The total number of 
priority schools in a state must be at least 5 percent of the Title | schools in the state. 


e Focus school: A Title | school that is contributing to the achievement gap in the state and, at the 
high school level, a Title I- school that has the largest within-school gaps in graduation rates. A 
Title | high school with a graduation rateless than 60 percent over a number of years must be 
identified as a focus school if it is not identified as a priority school. The total number of focus 
schools in a state must equal at least 10 percent of the Title | schools in the state. 


* A state also may include non-Title I-participating schools in its differentiated recognition, accountability and support systems; 
however, “Title |, Part A funds may be used only in Title I-participating schools. Although the Department will consider 
request for a waiver of ESEA section 1113(a)(3)-(4) and (c)(1) that would allow an LEA to use Title I, Part A, funds in Title I- 
eligible schools that are included within the definition of priority schools (i.e., Title |-eligible high schools that have had a 
graduation rate below 60 percent over a number of years), the Department does not intend to expand the scope of that 
waiver to include other Title I-eligible, but not participating, schools. (Added January 5, 2012)” (U.S. Department of 
Education, 2012b, p. 41). 
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e Other Title | schools: Other low-performing Title | schools, besides identified priority and focus 
schools, that are not making progress against established targets in improving student 
achievement and narrowing achievement gaps. 


e Reward school: A Title | school identified as among the highest performing Title | schools in the 
state with respect to the absolute performance on statewide assessments or graduation rates of 
all students and all subgroups of students, or a high-progress school that is among the top 10 
percent of Title | schools in the state with respect to progress on statewide assessments and, at 
the high school level, graduation rate. 


The purpose of this study was to explore state and district processes, successes and challenges, as well 
as school-level experiences associated with the early implementation of the accountability provisions of 
ESEA flexibility. The analysis of interviews and document data yielded the following findings. 


State Rationales for Accountability Under ESEA Flexibility. State officials in all 12 states described ESEA 
flexibility as an opportunity to address perceived deficiencies of the No Child Left Behind (NCLB) Act. At 
the same time, for most states in the study, accountability under ESEA flexibility built on or continued 
states’ existing policy reform efforts. 


Features of Accountability Under ESEA Flexibility. Five of the 12 states in the study used a combined 
subgroup as part of their accountability system under ESEA flexibility. Although some early critics of ESEA 
flexibility expressed concern that combined subgroups could mask the performance of some subgroups 
(Education Trust, 2012), the majority of state and district interviewees did not share these concerns. 


Although states are required to specifically identify priority, focus and reward schools, they are also 
required to ensure LEAs provide differentiated supports and interventions for other Title | schools that 
are not identified in one of these categories and who fail to meet achievement and graduation rate 
targets for subgroups of students. Eight of the 12 states also used a third improvement category, 
beyond priority and focus schools, for other low-performing schools that are not making progress in 
student achievement or narrowing student achievement gaps. The officials from these states reported 
developing a specific category for this group of other Title | schools to monitor and support schools that 
might not otherwise meet the criteria for priority and focus school designations. 


Communication and Understanding of Accountability Under ESEA Flexibility. The interviewed state and 
district officials described efforts to disseminate information and build local-level understanding of state 
accountability and support systems. Officials from two states and seven districts reported challenges 
related to these communication efforts, including effectively translating the complexities of the system. 
Among the Title | school principals interviewed for the study, perceptions of state and district 
communication efforts and understanding of state accountability and support systems were mixed. 
Principals in four of the six states from which the school sample was drawn did not raise concerns about 
the information they had received from their states and districts and consistently reported that they had 
a good understanding of their state accountability systems. Nearly all of the interviewed principals in the 
other two states indicated an overall lack of understanding of their state’s accountability system and 
expressed concern with the communications they had received from their state about the system. 
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Perceptions of State Accountability Designations. The majority of district respondents did not voice 
concerns about their state’s accountability designations and the measures that were being used to 
identify low-performing schools. However, officials from eight of the 22 districts pointed to perceived 
deficiencies related to ways in which states were measuring achievement gaps, how those metrics failed 
to capture the academic progress of English learner (EL) students, whether the reformulated student 
achievement measures were targeting the right schools or whether the changes to the system would 
improve student outcomes. 


The interviewed school principals consistently reported an appreciation for the intentions behind their 
state’s accountability system under ESEA flexibility. They perceived these systems as aiming to provide 
school performance targets that are clear and realistic and acknowledge intermediate progress. At the 
same time, the respondent principals varied in their perceptions of whether the criteria underlying their 
state’s system were valid or accurate measures of school performance. The interviewed principals in 
three of the six states perceived their state’s school designation criteria as more valid measures of 
school performance than the criteria that were used in the previous system. Respondent principals in 
the other three states expressed concerns about the criteria their states were using for determining 
school designation status under ESEA flexibility. 


Interventions and Improvement Strategies for Low-Performing Districts and Schools. The interviewed 
state and district officials reported that low-performing districts and schools were typically allowed to 
select their own interventions, but that their states had established guidance to help inform these 
decisions. The most frequently reported intervention strategies among the interviewed principals of 
priority, focus and other low-performing Title | schools included creating a culture of targeted data use, 
aligning instruction with standards, extending the learning time in core content areas, providing 
teachers with professional development and promoting a more positive school climate. 


Supports and Challenges for School Improvement Strategies. The state officials who were interviewed 
for this study described providing support to low-performing schools through the use of regional 
technical assistance organizations, external support providers and early warning systems. Ten of the 13 
priority and focus school principals who were interviewed described receiving more frequent or more 
intensive support from their districts under ESEA flexibility than in previous years, particularly around 
data use and interpretation and mathematics and reading instruction. Despite these supports, principals 
from 20 of the 25 schools in the sample described a variety of factors that challenged their improvement 
efforts, including limited resources; staff turnover; low staff morale; and testing requirements for ELs 
who are subject to the testing requirements for all students as well as to any state- and district- 
mandated tests for the EL subgroup. 
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State and District Capacity to Support School Implementation of ESEA Flexibility Provisions. Among 
the 12 state respondents in the study, eight described at least a modest burden associated with 
implementing the differentiated recognition, accountability, and support systems under ESEA flexibility, 
and another three respondents described a substantial effort but did not describe the level of effort as a 
burden. State officials reported implementing strategies to build capacity to implement the 
accountability and support provisions, which included cross-agency collaborations, developing staff 
expertise, partnerships with districts, structural changes within the SEA and consolidating multiple 
funding streams. Among the interviewed district officials, about half indicated that implementing the 
new state accountability systems under ESEA flexibility had required effort but had not been 
burdensome. At the same time, officials from seven of the 22 districts in the sample reported 
insufficient staff and expertise to provide the level of support they believed was needed to improve low- 
performing schools in their districts. 


viii 


Implementing Accountability and Supports Under ESEA Flexibility 


Introduction 


This study’s purpose was to explore the early implementation of state-differentiated recognition, 
accountability and support systems under ESEA flexibility. The Elementary and Secondary Education Act 
of 1965 (ESEA) is the nation’s key policy driver for elementary and secondary education, shaping federal, 
state and district efforts to promote effective school systems and improve educational outcomes, 
particularly for students in high-poverty schools. The most recent reauthorization of ESEA (the No Child 
Left Behind Act of 2001, or NCLB) called for landmark changes. Among its many provisions, NCLB 
required states to adopt challenging academic content standards and aligned assessments, created a 
system of accountability for districts and schools, introduced new teacher quality requirements, and 
outlined specific consequences for not meeting annual performance objectives. 


As time passed after the implementation of NCLB and Congress did not reauthorize ESEA as expected, 
policymakers expressed concerns that some of the law’s provisions might not align with evolving policy 
and practice. In the absence of a reauthorized ESEA, the Obama administration announced a temporary 
measure known as ESEA flexibility, designed to give states flexibility to advance innovative education 
reform efforts by leveraging the Secretary of Education’s authority to grant states and districts waivers 
from certain provisions of NCLB. 


Specifically, ESEA flexibility offers states flexibility with regard to how states set annual measurable 
objectives (AMOs), whether or not they continue to calculate adequate yearly progress (AYP), designate 
low-performing schools and districts, implement interventions, use section 1003(a) funds?, determine 
transferability of ESEA funds and implement improvement plans for highly qualified teachers. States that 
were interested in receiving such flexibility submitted a request to the Department detailing the ways 
the state’s reform plan would (1) ensure college- and career-ready expectations for all students; (2) 
implement state-developed differentiated recognition, accountability and support systems; (3) support 
effective instruction and leadership; and (4) reduce duplication and unnecessary burden. The 
Department offered five submission windows for state ESEA flexibility requests, with deadlines in 
November 2011, February 2012, September 2012, February 2013 and April 2014. As of June 2015, the 
Department had received flexibility requests from 46 states, the District of Columbia, Puerto Rico and 
the Bureau of Indian Education. Among these 46 states, the Department had approved the requests for 
43 states, the District of Columbia, and Puerto Rico.? The sample of districts and schools in this study’s 
analyses were drawn from the group of states approved for ESEA flexibility in the first two windows 
(referred to in this report as Window 1 and Window 2). 


2 States must use these funds to support Focus and Priority schools and if approved via an additional waiver, may use them to 
support interventions and support in other Title | schools. 

3 The flexibility requests of three states (lowa, Nebraska, and Wyoming) and the Bureau of Indian Education were in review 
status as of June 2015. Also, although Washington’s flexibility request had earlier been approved, the state’s flexibility was 
not extended through 2014-15. 
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Study Sample 


The primary data sources for this study were ESEA flexibility policy documents and telephone interviews 
with key officials in selected state educational agencies, local education agencies (LEAs), and schools. 
The study team reviewed state ESEA flexibility requests and other policy-related materials on state and 
district websites. The team also conducted semi-structured interviews with state and district officials 
who held primary responsibility for accountability and support systems under ESEA flexibility, and 
principals of Title | schools that had received priority, focus or reward designations or were otherwise 
designated as a low-performing school for the 2012-13 school year. The interview participants included 
officials representing 12 states, 22 (of 24 selected) districts, and 25 (of 36 selected) schools. 


| _ —— 7 i The study team purposefully selected the 12-state sample to include state 
Li (& * variation in (1) time of ESEA flexibility approval—either Window 1 or 
% % / — at Window 2 states, (2) region and size, (3) receipt of Race to the Top grants, 
: ao (4) approach to annual measurable objectives and (5) use of combined 
) byden subgroups. The selected states included Colorado, Kentucky, Maryland, 
Massachusetts, Minnesota, Missouri, New Jersey, New Mexico, New York, 
Oklahoma, Tennessee and Washington. 


The study team selected the district and school samples from the 12-state sample to reflect variation in 
(1) the number of reward, focus, priority and other low-performing schools Title | schools; (2) size and 
urbanicity; and (3) demographic diversity. For the district sample, the district’s intent to apply as a Race 
to the Top applicant also was considered and treated as a rough proxy for capacity to implement 
accountability system reforms. The school sample was also purposefully selected to reflect variation in 
ESEA flexibility accountability designation, school size, district size, student demographics, urbanicity 
and geographic location. The final district and school samples included 22 district participants and 25 
school participants, respectively. Within the school sample, seven of the schools were designated 
priority schools, six were designated focus schools, six were designated reward schools and six were 
other low-performing Title | schools for the 2012-13 school year. 


Study Questions 


As previously noted, this study’s purpose was to explore the early implementation of ESEA flexibility 
from the state, district and school-level perspectives. The study’s primary questions focus on the 
experiences, including the challenges and successes, of officials at these three levels and asked the 
following questions: 


1. Approaches to accountability and supports under ESEA flexibility. What prior experiences and 
rationales guided state approaches to accountability and support under ESEA flexibility, 
including states’ selection of AMOs? 


2. Features of accountability under ESEA flexibility. What are the primary components of state 
accountability and support systems under ESEA flexibility? 


3. Communication of state accountability and supports under ESEA flexibility and school-level 
understanding of state systems. What modes of communication did states and districts use to 
inform and build local-level understanding of state systems under ESEA flexibility? To what 
extent were these communication efforts perceived as effective? 
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4. Perceptions of the state accountability designations under ESEA flexibility. How did local-level 
officials perceive state accountability systems under ESEA flexibility, including the criteria being 
used to identify low-performing schools? 


5. Interventions and improvement strategies for low-performing schools and districts. What 
approaches are states taking to identify and intervene in low-performing districts? What 
improvement strategies did principals report implementing? 


6. Supports and challenges for school improvement strategies. What structures or mechanisms 
were states and districts using to support low-performing schools, and how did principals 
perceive these supports? 


7. State and district capacity to support the implementation of ESEA flexibility provisions. How 
did states and districts perceive their capacity to implement ESEA flexibility provisions, and what 
factors did states and districts report as challenging and supporting their implementation efforts? 


Data Collection and Analysis 


This report’s findings are based on telephone interviews with state, district and school officials, as well 
as on a review of state and district websites and relevant policy documents that were available online. 
The telephone interviews and document reviews were conducted in late 2013 and early 2014. The 
interviews were conducted using semi-structured protocols to balance questions that were sufficiently 
uniform to yield comparable data across states and districts but varied enough to allow for documentation 
and exploration of the unique circumstances within each state, district and school. Therefore, the 
structure of the interviews and the specific questions asked of the officials varied across respondents. 


The analytic process included multiple steps designed to promote timely, reliable and rigorous analyses. 
The interviews were recorded, transcribed and reviewed for accuracy. The study team then compiled 
data from interviews and document reviews in a Web-based platform, or data repository, which divided 
interview topics into sets of open- and closed-ended questions. The team developed decision rules and 
evidence thresholds that the analysts applied to identify themes and determine how to classify states, 
districts and schools into theme categories. The coding and key findings were independently reviewed 
by analysts to promote inter-coder agreement on state, district and school classifications. 


This report most often identifies states by name when describing specific policies and practices because 
state policy is a matter of public record and there is no reason to mask a state’s identity. However, when 
describing a comment that reflects the personal opinion or perspective of a state official—particularly if 
there is any possibility that this could be construed in a negative light—this report omits the state name. 
In all cases, district and school names have been omitted. 
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Study Limitations 


Readers should note some limits to the interpretation and generalizability of the study. First, as noted 
previously, the interviews were semi-structured, allowing some variability in the questions asked across 
respondents. As a result, there might be an undercount of states, districts or schools on some topics 
because the interviewer did not ask a state, district or school official about a particular topic. Second, 
state and district data were limited to individuals or groups of individuals appointed by state and district 
leaders to participate in interviews and so reflect these individuals’ knowledge and perspective; 
similarly, school-level data were collected from the principals who were leading the school at the time of 
data collection and reflect solely their knowledge and perspective of the topics about which they were 
asked. Third, the study included only a subset of states with approved ESEA flexibility requests. The 
Department approved 35 states’ ESEA flexibility requests in Window 1 and Window 2, and this study 
includes information collected from principals at 25 schools and officials in 22 (of 24 selected) districts 
within 12 of these 35 Window 1 and Window 2 states. Moreover, the states, districts and schools, 
although diverse, were not randomly sampled and are not representative. For these reasons, the data 
collected and the reported findings are not generalizable to all states, districts and schools. 


Readers should also note that in reporting findings related to states, we have masked the identity of the 
states except when describing a policy that is already in the public domain. 
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Findings 


Approaches to Accountability and Supports Under ESEA Flexibility 


ESEA flexibility provides states with autonomy to design state accountability systems that align with 
their individual state context, priorities and reform histories using three guiding principles: (1) establish 
college- and career-ready expectations for all students; (2) develop differentiated recognition, 
accountability and support systems; and (3) support effective instruction and leadership. States, districts 
and schools are required to expect improvement in student outcomes but may determine their own 
goals, interim targets and plans for achieving those goals rather than measuring against a uniform 
objective of 100 percent proficiency by 2014. Indeed, the state officials interviewed in all 12 states in the 
sample described ESEA flexibility as an opportunity to address perceived deficiencies of NCLB, anda 
review of state ESEA flexibility requests along with the interview data demonstrate that states often had 
an underlying theory of action that guided the development of their systems of support for low- 
performing schools. These underlying theories of action implied a set of assumptions about the reasons 
schools were low performing and leveraged points to build school capacity and foster improvement. 
Furthermore, all of the interviewed state officials reported that their states had already started down a 
path of refining their approach to school and district accountability when they applied for ESEA flexibility 
to address perceived problems with the law’s requirements. 


Among the perceived problems with NCLB requirements reported by state officials, nine officials 
reported that too many schools were being identified for improvement. Officials from nine states 
reported that, each year, as more and more schools were identified for improvement, the designations 
became less meaningful. For example, officials from four states expressed concern about the lack of 
differentiation among low-performing schools designations, making it difficult to provide targeted 
supports. In addition, officials from three states indicated that the over-identification of schools led to a 
compliance-driven culture and strained state capacity to support school improvement. In contrast, the 
officials reported that, under ESEA flexibility, their states were able to designate a smaller number of 
schools for school improvement and to target their supports accordingly. In the words of one official: 


We’re really able to focus more on the schools in our state who are really underperforming 
instead of identifying almost every school in our state as “does not meet AYP [adequate yearly 
progress].” And that’s been a huge focus for us because we’re able to concentrate our work 
specifically in those buildings, schools, and districts that are significantly underperforming. So we 
now have 170 buildings to focus on instead of almost 2,000 buildings, and that’s significant. 


State officials reported that NCLB pilot initiatives were influential in the design of their state 
accountability systems under ESEA flexibility. Through NCLB pilot initiatives dating back to 2005, eight 
of 12 states in the study sample had modified their measures of proficiency or designations of low- 
performing schools. Massachusetts, Minnesota, New York and Oklahoma received flexibility to develop 
a performance index with multiple measures to determine adequate yearly progress. Colorado, 
Minnesota, Missouri and Tennessee were approved to measure proficiency using student achievement 
growth as part of the NCLB growth model pilot. Maryland and New York had flexibility to develop 
differentiated accountability systems in which additional levels of intervention allowed for more 
nuanced targeting of intervention and support. 
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These officials reported that their participation in these pilot initiatives was one of the factors that 
influenced the design of their state accountability systems under ESEA flexibility. For example, a New 
York state official explained: 


We have had a performance index for English/language arts and mathematics in New York State 
since the inception of NCLB. ... So this was something that was kind of baked into the New York 
state system. And then when we moved to our ESEA waiver, we took the concept of the 
performance index, but we made several changes to it under the waiver. 


Exhibit 1 summarizes state participation in federal and state policy initiatives prior to ESEA flexibility. 


Exhibit 1. Federal and State Policy Activities Prior to ESEA Flexibility Among the 12 States 
in the Study Sample 


Recent State 


NCLB NCLB NCLB Policyintroducing: 
. : Race to P Introduced 
Performance Growth Differentiated or Reforming . 
State : a the Top ie Policy 
Index Pilot Model Accountability Accountability 
é ‘ Awardee Through 
(2007) Pilot Pilot System Other pasislation 
Than NCLB € 
Colorado 2008 Round III X 2009 
Kentucky Round III X 2009 
Maryland 2008 Round Il 
Massachusetts X Round II X 2010 
Minnesota X 2008 X 2012 
Missouri 2007 X 2011 
New Jersey Round III 
New Mexico X 2011 
New York X 2009 Round Il X 2009 
Oklahoma X 2011 
Tennessee 2005 Round | X 2010 
Washington X 2010 


Source: U.S. Department of Education website, state department of education websites and interview data from state officials. 


Features of Accountability Under ESEA Flexibility 


State educational agencies developed differentiated recognition, accountability and support systems 
under ESEA flexibility. State systems under ESEA flexibility were required to set AMOs and to identify 
priority, focus and reward schools. ESEA flexibility also includes the flexibility for states to create and use 
other accountability designations to identify schools beyond reward, priority and focus schools that met 
certain criteria. Priority schools are Title | schools that are among the lowest performing schools in the 
state, focus schools are Title | schools with the greatest achievement gaps or in which subgroups are 
furthest behind, and reward schools are highest performing and high-progress schools. Under the 
provisions of ESEA flexibility, states are required to ensure that priority, focus, and any other Title | 
schools that are not making progress or narrowing achievement gaps receive interventions and supports. 
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Identifying other low-performing Title | schools 


Although states are required only to identify priority, focus and reward schools under ESEA flexibility, 
eight states in the study sample established a specific accountability designation category for other 
Title | schools to facilitate their efforts to provide such accountability and support provisions. 

ESEA flexibility guidance requires states to describe how state systems will provide incentives and 
support improvement in other Title | schools that are not making progress or narrowing achievement 
gaps and not designated as priority or focus schools. To satisfy this component of ESEA flexibility, 
officials from eight of the 12 interviewed states (Colorado, Maryland, Massachusetts, Minnesota, New 
Mexico, New York, Oklahoma and Washington) developed a specific category of low-performing schools 
to monitor and support other Title | schools (and in some cases non-Title | schools) that might not 
otherwise meet the criteria for priority and focus designations; interviews with state officials highlighted 
state priorities to support all schools in need of assistance. In the words of a state official from Washington: 


There’s that part in the ESEA flexibility request that talks about serving other low-performing 
Title | schools, and ... we’re using the word emerging [to describe other low-performing schools] 
because they’re certainly headed in the right direction. ... With some support, we hope that they 
grow and don’t end up on the focus list. 


See Exhibit Al for the definitions these eight states with specific accountability designations for other 
low-performing schools used to identify and categorize such schools. 


At the district level, local officials used their own classifications of other Title | low-performing schools 
to monitor and support schools that were not identified as focus and priority schools but that were 
still demonstrating insufficient progress toward improving student achievement or narrowing 
achievement gaps. Among the eight districts in the four states that did not specifically have a 
designation category for other low-performing Title | schools, three districts used their own system to 
identify other low-performing Title | schools. For example, officials in one district were concerned that 
not all schools that needed targeted support were being identified for improvement under the focus 
and priority designations. 


Setting AMOs 


Among the three ESEA flexibility options for setting new AMOs, eight of the 12 states in the sample 
chose to devise their own goals for measuring student achievement. With respect to developing AMOs 
under ESEA flexibility, states were given the opportunity to replace the goal of 100 percent proficiency 
by 2014, which was set for all states under NCLB. States had flexibility to select among three options to 
set new ambitious, yet achievable, annual targets for school improvement at the state, district and 
school levels. States could choose option A (to halve the percentage of students—all students and all 
subgroups—not proficient in six years, using proficiency rates in reading and mathematics), option B (to 
achieve 100 percent proficiency by 2020, using average statewide proficiency rates in reading and 
mathematics), or option C (to devise their own ambitious yet achievable goals, using an educationally 
sound method, for measuring student achievement that are similarly ambitious to the AMOs that would 
result from using option A or B). 
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Eight of the 12 states in the sample (Colorado, Kentucky, Massachusetts, Missouri, New Mexico, New 
York, Oklahoma and Tennessee) selected option C, to develop their own methods for setting AMOs. The 
officials from these eight states described a variety of reasons for choosing option C. Missouri, New York 
and New Mexico officials favored option C because they wanted to align the AMOs with components of 
their state system for assessing performance. Colorado, Kentucky, New Mexico and Oklahoma officials 
said that they wanted to be able to include additional measures beyond reading and mathematics 
AMOs. For more information on these states’ approaches under option C, see 
http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html where the most up-to-date flexibility 
requests for each state and related documents are accessible. The remaining four states in the sample 
(Maryland, Minnesota, New Jersey and Washington) chose option A, to halve the percentage of 
students not proficient in six years. The interviewed officials from these four states indicated that they 
perceived this approach as more simple and practical than options B and C. None of the states in the 
sample selected option B, 100 percent proficiency by 2020. Several state officials reported that they did 
not favor this option because they believed it would continue a policy that was not working. In the 
words of one New Jersey official: 


Option B would have been almost the simplest thing to do... but it felt like kicking the can. It 
didn’t help us in inspiring all schools to address their weaknesses because it just literally 
extended the time frame for a system that we already felt wasn’t working well. 


Using combined subgroups 


Five of the 12 states in the study sample used a combined subgroup as part of their accountability 
system under ESEA flexibility. One of the central components of accountability under NCLB was 
disaggregating data by individual subgroups. ESEA flexibility provided guidance that required states to 
continue using these individual subgroups but allowed state educational agencies to create combined 
subgroups that would consolidate into a single group a set of NCLB subgroups’—for example, racial and 
ethnic groups, economically disadvantaged students and/or students with disabilities—when identifying 
low-performing schools. As presented in Exhibit 2, five of the 12 states in the study sample (Kentucky, 
Massachusetts, Missouri, New Mexico and Oklahoma) chose to use a combined subgroup as part of 
their accountability system under ESEA flexibility. 


Exhibit 2. _ Composition of Combined Subgroups in Five States With Subgroups 


State Students Included in Combined Subgroup 

Kentucky Historically disadvantaged race/ethnic students (African American, Hispanic, Native 
American), students with disabilities, economically disadvantaged, English learners 

Massachusetts Students with disabilities, current and former English learners, and economically 
disadvantaged students 

Missouri Black and Hispanic students, low-income students, students with disabilities, and/or 
English learners 

New Mexico Students achieving below the 25th percentile in reading and mathematics 

Oklahoma Students achieving below the 25th percentile in reading and mathematics 


Source: State department of education websites, state ESEA flexibility approved requests and interviews with state officials. 


“We use the term NCLB subgroups to distinguish the subgroups that were specified in the 2001 reauthorization of ESEA. 
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Although the Department’s ESEA flexibility guidance requires states that use a combined subgroup to 
continue to include the performance of all ESEA subgroups in its accountability system, specifically in 
reporting AMOs, and to demonstrate that the composition of the subgroup is educationally sound, one 
potential concern with this approach is that higher performing groups of students could mask the 
underperformance of other groups of students. This concern was highlighted by peer reviewers of the 
state’s ESEA flexibility requests and through an analysis of the accountability provisions implemented by 
the first 11 states that were approved to implement ESEA flexibility (Education Trust, 2012). Among the 
five states in this study’s sample that were using combined subgroups, officials from four did not express 
concern that this approach would draw attention away from individual subgroups. In addition, officials 
from eight of the 12 districts in this study’s sample that were located in the five states using combined 
subgroups did not express concern about this approach. However, officials in four of the 12 districts did 
express some concerns that the needs of some students might be masked. As one official commented: 


I’m happy for the increased accountability. But sometimes when you lump a lot of groups 
together, it’s easy to—particularly for educators who are not well trained and have issues with 
disadvantaged and English learners—it’s easy to get lost in the fog of all that. 


Recognizing reward schools 


State officials from all 12 states in the sample reported providing opportunities for financial rewards 
for reward schools, and these officials reported identifying reward schools for reasons that included 
recognition for improvement and achievement gap closure. Among the 12 states in the sample, 
officials from six states (Colorado, Kentucky, Maryland, Massachusetts, New York and Washington) 
reported that the ESEA flexibility request prompted them to make revisions to existing formulas or 
criteria in order to identify reward schools, such as including achievement gap closure as a factor and/or 
expanding reward school designations to include improving or high-progress schools. Three state 
respondents (Massachusetts, New Mexico and Tennessee) stressed the importance of recognizing low- 
performing schools that have demonstrated growth. As the official from New Mexico noted, a result of 
designating reward schools was to establish examples of successful school turnaround across the state 
“to spread around the positive results” by connecting practitioners from low-performing schools with 
leaders and staff who have made progress in school turnaround. 


Officials from all states in the sample reported providing opportunities for monetary awards to 
identified reward schools, but the forms of financial rewards and conditions for payment varied by state. 
Officials from six states (Colorado, Kentucky, Maryland, Minnesota, New Jersey and New Mexico) 
reported a policy of direct monetary bonuses to schools. Additionally, in three of those states (Kentucky, 
Minnesota and New Mexico) and in five others (Massachusetts, New York, Oklahoma, Tennessee and 
Washington), reward schools were eligible to compete for grants to disseminate information on best 
practices or to participate in compensated partnerships with other schools. In addition to the variance in 
rewards across the states, most states structured financial rewards to be contingent on available 
funding, and officials in six states reported funding shortfalls that limited rewards. 
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Apart from monetary awards, state officials described other forms of recognition for reward schools. 
Officials from all 12 states described public recognition for reward schools, including public praise in 
press releases, state publications, and on the state educational agency website; public praise at state 
events and conferences or through school visits from state officials; certificates, awards, or letters of 
recognition presented to the school or staff; and permission for the school to use reward status or 
official state logos for promotion of the school. State officials in four states (Maryland, New Mexico, 
New York and Oklahoma) also described exemptions from regulatory requirements or greater autonomy 
to reward schools (see Exhibit A2 for a table presenting the nonfinancial rewards offered to reward 
schools by the 12 states in the sample). 


Among the six reward schools in the sample, one school principal reported receiving formal, but 
nonmonetary, recognition from the state; none of the reward school principals reported receiving 
monetary rewards or compensation. In general, the principals of the reward schools in the sample 
reported that their reward status was not accompanied by any special recognition, such as bonuses or 
other incentives, from their respective states. One of the interviewed principals indicated that he was 
not made aware of his school’s reward status until being contacted for this study. The principals of the 
other five reward schools reported being aware of their school’s reward designation, but only one of 
these five principals reported receiving any formal recognition from the state. This recognition came in 
the form of a plaque noting the school’s reward status. Another principal stated that she assumed the 
school’s reward status was published in the newspaper, but she had not been notified if it was. Another 
principal explained that the school’s status was noted in Title | quarterly reviews and annual reports but 
had not been publicized in any other ways. Overall, it appeared that reward school principals, while 
pleased with the designation, did not perceive the accountability systems under which they had received 
reward status as particularly focused on emphasizing the improvements or successes of reward schools. As 
one principal remarked, “It seems like ... we’re punitive when people are not doing well. ... It would be nice 
to be able to recognize teachers and be able to recognize some really amazing hard work.” 


Communication and Understanding of Accountability Under ESEA Flexibility 


The development and implementation of meaningful accountability systems may be contingent on 
school leaders and stakeholders having a strong understanding of the system. Thus, states’ and districts’ 
abilities to communicate effectively with school leaders about the accountability systems may be critical 
to fostering the understanding and motivation at the school level that can mobilize successful and 
sustained turnaround of low-performing schools (O’Day, 2002). 
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State officials reported multiple modes of communication with local education leaders to disseminate 
information and build understanding of their new accountability systems. All of the state interview 
respondents indicated that their states’ communications with local education leaders and other 
education stakeholders often were provided in face-to-face settings such as working groups or training 
sessions on specific topics. Officials from 11 of the 12 states reported that they held periodic 
committees, advisory councils or commissions to provide ESEA flexibility updates and new information 
on an ongoing basis. For example, Washington’s state official reported that the state educational agency 
had established an Educational Opportunity Gap Oversight and Accountability Committee that brought 
representatives from 22 organizations together to discuss implementation challenges with the new 
system and the role of the state in assisting low-performing schools. This state official reported that this 
group was “a very, very robust and deliberate structure” for disseminating state and federal 
accountability information. Officials from six states reported using written communications, such as 
practice guides, newsletters, press releases and resource documents, to communicate information on 
the new accountability system. 


District officials generally reported that they relied on existing structures rather than new structures 
for disseminating information to and obtaining feedback from stakeholders on ESEA flexibility. 
Officials in 15 out of 16 districts who reported having feedback structures stated that they used existing 
systems, such as parent advisory groups or school-site planning meetings, to explain aspects of the new 
accountability system and to gather any feedback on implementation to date. One district official 
reported having two separate Title | meetings each year and formally incorporating discussion of the 
new accountability system in those meetings. 


Officials from two states and seven district officials described challenges related to translating the 
state accountability and support system to schools and the community to support local-level 
understanding of the system. Most of the interviewed state officials did not mention challenges related 
to communicating information about the state system of accountability and supports under ESEA 
flexibility; however, officials from two states specifically described such challenges. For example, one of 
these state officials indicated that the state sent “[district leaders] information all the time and we say, 
‘Share this with your schools.’ And they never do. So we have had to start thinking about how we can 
get information down specifically to school leaders because the district doesn’t always send it along.” 


Among the 22 districts in the sample, officials in seven states suggested that the new systems were too 
complex and expressed frustration related to trying to translate the new system to teachers, parents, 
and students. One district official stated, “As an administrator, to be able to simplify the understanding 
of the language and the accountability system is quite challenging.” Another district official shared a 
specific example of the struggle the district has with helping principals understand the metrics necessary 
to develop an actionable plan: 


| had school leaders asking me this morning, “So | got 2.9 points for schoo! growth. How do |! 
figure out what my target is for school growth so that | can actually have some measurable goals 
that | can deliver to my teachers as to where we want to go with school growth?” There’s no 
comparison on measuring yourself as to basically how to improve, if that makes sense. 
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In addition, three of the interviewed district officials reported feeling that the responsibility for 
communicating the new system to educators and the public was not their own but rather the state’s 
responsibility. One district official explained: 


[District staff] don’t feel that it is their job, and they don’t feel equipped to do it anyhow because 
they don’t completely understand it themselves. ... And in trying to sell the system, we don’t feel 
that’s our responsibility, and the state doesn’t feel like it’s our responsibility either, quite frankly. 


Another district official reported that the district had chosen to explain only the parts of the accountability 
system that were relevant to the district, rather than the system overall. The official stated: 


When I’m thinking back three years, | did not see and | don’t think our district saw our role to 
explain this accountability system. We thought it was the state’s role. So that does not mean we 
don’t go public with the results or explain those results. And we have like little press conferences. 
But this particular accountability system is pretty hard to understand—more complicated and 
convoluted than AYP. 


At the school level, all of the interviewed principals in four of the six states in the school sample 
consistently reported that they had a good understanding of their state accountability systems. These 
principals attributed their understanding to state and district communication efforts. The principals in 
one state consistently praised the communication they had received from the state and their districts 
about the system and the implications of their accountability designation. Two of these principals 
commended the direct lines of communication they had with state staff and the efforts state staff had 
made to make themselves available to answer questions by phone, webinar or onsite. One of these 
principals reported the following when asked about whether the state’s communications about the 
accountability system under ESEA flexibility had fostered school-level understanding of the system: 


I’m understanding it [the school accountability rules] more, and we really use the data with that 
lens of looking at proficiency, growth, and achievement gap. We talk about it with our parents, 
teachers, and my boss, and other folks at the district level talk to us about it. It provides a shared 
language. 


As another example, the three interviewed principals in a different state praised the meetings their 
districts had held with them to explain the school accountability criteria and the calculations on which 
the school designations were based. One of these school principals explained: 


We have a data person who works with us and an accountability person who comes out and 
makes sure we understand [our designation] ... we in turn ask questions ... then, they in turn 
make sure we have an understanding so we can relay understanding, so we can relay 
information in a way that is understood by our staff. 
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In two other states, seven of the 11 interviewed principals described an overall lack of understanding 
of their state’s accountability system and expressed concerns about the communications they had 
received about the system. Indeed, these principals expressed frustration about the communication not 
received about their state’s differentiated recognition, accountability, and support system. As 
mentioned, a principal from one of the states reported receiving no information or notification of his 
school’s reward designation, stating that he had not been made aware of his designation until contacted 
for this study. Although all other principals from these two states were aware of their school’s 
designation, they consistently reported that their states had provided insufficient information and 
resources to support their full understanding of the system. According to one principal, the state had 
disseminated a handbook with guidelines for how the school accountability designations would be 
calculated but did not provide any accompanying trainings or workshops to further support school-level 
understanding or correct interpretation of the guidelines. In addition, the principal reported that the 
state “kept changing that guide, over and over. ... | would look at it one way, they would change the 
criteria, and | would have to look at it another way.” 


Perceptions of the State Accountability Designations Under ESEA Flexibility 


As noted previously, under ESEA flexibility, states are required to identify priority, focus and reward 
schools. States also are required to identify and provide support to other Title | schools that are not 
demonstrating progress, but are not identified as priority or focus schools. These other Title | schools 
include schools in which subgroups of students miss AMOs or graduation rate targets or both over a 
number of years. States have the option to develop a specific school accountability designation for 
these schools. While ESEA flexibility includes these provisions, states with approved flexibility requests 
have been provided autonomy to design state accountability systems that align with their own 
circumstances, priorities and reform histories within the parameters of the flexibility principles. One 
intention of this flexibility is to provide states with the opportunity to develop meaningful systems that 
stimulate local change in education policy and practice, ultimately resulting in improved student 
outcomes (U.S. Department of Education, n.d.). However, the extent to which state accountability 
systems under ESEA flexibility can enable and motivate improvement is likely dependent on key 
stakeholders, including school-level personnel perceiving the accountability designations as accurate 
and meaningful (O’Day, 2002). 


In three of the six states in the sample, all 10 of the interviewed school principals perceived the 
criteria that their states were using to identify low-performing schools under ESEA flexibility to be 
more valid measures of school performance than the criteria that were used in the previous 
accountability system. In these three states, none of the interviewed principals voiced major concerns 
about their state’s accountability system under ESEA flexibility. These principals generally described the 
accountability measures as more comprehensive and fairer than the measures used to assess school 
performance under previous state systems. In particular, they emphasized their appreciation for 
measures that included proficiency, growth and achievement gaps rather than a single proficiency 
measure. As one principal stated, “Looking at growth for students who are not performing is a good 
thing and can help motivate staff and help staff celebrate success. Just looking at proficiency can be 
demoralizing.” Even schools that had received a priority designation viewed the designation as less 
punitive than the designations they had received under the former state system. One priority school 
principal stated: 
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The priority school environment allowed us to demonstrate a growth factor. That was a big part 
of why we were able to move designations, because we met our growth goals. Obviously, we’re 
totally aware that we have to continue to meet those goals and that the bar raises, but what it’s 
done for us is that it’s not made the standards so high that it feels impossible. 


Eight of the 15 interviewed school principals in the other three states expressed concerns about the 
criteria their states were using to make school designation determinations. Despite reporting that they 
had a good understanding of their state’s differentiated accountability system, four principals in one 
state expressed reservations about the way in which the state ranked schools in comparison with each 
other based on student achievement growth. 


In another state, all six interviewed principals voiced support for the state accountability system’s focus 
on student achievement growth rather than just proficiency rates, but these principals perceived the 
system as punitive in implementation. These principals expressed a desire for the system to focus more 
strongly on the successes schools were achieving. In one state, two of the five interviewed principals 
raised concerns about the metrics that were being used to identify schools, including concerns about the 
accuracy of the data and whether the metrics were valid measures of performance. According to one 
principal, the school had received a designation report that included findings that resulted in staff 
feeling “confused or discouraged by the ratings.” 


The majority of district respondents did not voice concerns about their state’s accountability 
designations and the measures that were being used to identify low-performing schools. However, 
officials from eight of the 22 districts pointed to perceived deficiencies. The officials from these eight 
districts raised concerns about the ways in which states were measuring achievement growth (three 
districts), ways in which the metrics continued to fail to capture the academic progress of EL students 
(three districts), whether the reformulated student achievement measures were targeting the right 
schools (two districts), as well as a general critique that the measures were “less informative” (two 
districts). (Districts may fall into more than one category.) 


Some state officials remarked on the benefits of ESEA flexibility with respect to aligning dual systems 
of state and federal accountability. At the same time, other officials described continuing challenges 
related to aligning their accountability systems. One of the problems with NCLB that states sought to 
address under ESEA flexibility was the existence of a dual system of federal and state accountability. For 
example, officials in 10 of 12 sampled states reported having had a statewide accountability system with 
school or district designations that were different from those associated with NCLB accountability. The 
officials from four of these states specifically stated that aligning their state accountability system with 
the federal system was a prominent reason for applying for flexibility.” One Colorado official stated: 


We really wanted to create a single accountability system. We felt like our state system was ina 
second year of implementation and it was mature enough to carry the load for both state and 
federal. And so we pursued the waivers. 


One official reported that the state system and federal system had been growing farther apart in their 
determination of which schools were low performing, which was becoming increasingly problematic and 
confusing for districts and schools. 


: Study interviewers did not probe specifically on this point. Thus, it is possible that more states share this perspective but did 
not volunteer the information. 
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At the same time, in implementing the ESEA flexibility accountability provisions, some officials reported 
continued challenges in aligning their state and federal accountability systems. Officials from eight other 
states reported that they struggled to reconcile the designation of low-performing schools under the 
state accountability designations with the ESEA flexibility requirements for either priority or focus 
schools. These officials reported that the criteria for identifying priority or focus schools were 
problematic because the schools identified by the guidelines set in ESEA flexibility were often different 
from the schools identified as low performing using the state-devised index or grading system. 
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Interventions and Improvement Strategies for Low-Performing Schools and 
Districts 


The interviewed state and district officials reported that low-performing districts and schools typically 
were allowed to select their own interventions, but that their states had established guidance to help 
inform these decisions. Among the 12 states in this study, the organizations involved most often in 
determining interventions in priority schools were the district (eight states), school (six states) or 
regional organization (three states). In seven states, the decision was shared across multiple 
organizational levels. 


Box 1: Turnaround Principles 


Strong leadership 

Effective teachers 

Redesigned schedules for additional time 
Rigorous and aligned instructional program 
Use of data for continuous improvement 
Safe and healthy students 

Family and community engagement 


The states in the study sample often developed an array of interventions from which districts and 
schools could choose to support improvement; these interventions were typically based on turnaround 
principles established by the Department as part of ESEA flexibility guidelines (see Box 1) (U.S. 
Department of Education, 2012a, p. 2).° One state in the sample, for example, presented a “menu of 
interventions and supports” in its ESEA flexibility request from which districts and schools could select, 
including extended learning time, training for instructional leaders, the adoption of early college 
programs for high schools, and other instructional and behavioral interventions and supports. In six 
states, state education agencies were involved in district and school selection of interventions, either by 
reviewing plans or by working collaboratively with each district or school to identify appropriate 
interventions. These states required districts or schools to conduct needs assessments to guide the 
selection of their interventions. According to one state official: 


The state provides the parameters for which interventions are acceptable. The district in 
consultation with the school selects the actual intervention. The rationale is that the district is 
closer to the school ... and then the state doesn’t have the resources to select interventions for all 
schools. 


Respondents in 15 of the 22 districts representing eight states in the study sample provided information 
on their level of autonomy in selecting school improvement interventions. Officials from nine of these 
15 districts indicated that they have a great deal of latitude. Officials noted, however, that in some 
cases, intervention plans required state approval. When asked about the district role in identifying 
interventions, one district respondent explained: 


: Although these interventions were typically found to be consistent with turnaround principles, ESEA flexibility policy requires 
only priority schools to implement interventions that are aligned with turnaround principles. 


16 


Implementing Accountability and Supports Under ESEA Flexibility 


The district does have a lot of leeway ... which is really good. However, district plans, especially 
with schools that are priority improvement or turnaround, have to be reviewed by a state panel 
made up of members from [the state educational agency] and others from around the state 
trained to review plans. 


The intervention strategies most frequently described by principals included creating a culture of 
targeted data use, aligning instruction with standards, extending the learning time in core content 
areas, providing teachers with professional development and promoting a more positive school 
climate. The data collected from principal interviews did not allow for an assessment of the intensity 
with which their reported school improvement strategies were being implemented. Rather, the 
interviews focused on the strategies that the priority, focus and other designated low-performing Title | 
schools had selected and their rationale for doing so. 


The interviewed principals consistently described strategies that were informed by their data analyses 
and identified areas of weakness—specifically, improving student performance on reading and 
mathematics and closing achievement gaps between subgroups of students. Principals described varied 
strategies for addressing these needs, including extended learning time in these core subject areas, 
more frequent data collection and analysis to monitor gaps in achievement and progress among 
subgroups of students, and more closely aligned instruction with the standards and assessments. One 
elementary principal explained: 


We said, “We know the standards, we know what the state will be testing on, so let’s start 
there.” Let’s use the math[ematics] program to teach to the standards, but put the standards 
first. We developed mini-quizzes based on the standard the program was supposed to be 
teaching for a particular week. ... The teachers would use the results of these quizzes to 
determine whether to move on or reteach the standard students were learning. We did this in 
the tested grades for math[ematics], grades 3—5. And we saw an 11 percent gain in proficiency. 


Principals also described providing professional development supports to enhance teachers’ 
instructional practices and to foster collaboration among teachers around data use and instruction. For 
example, the principals of two schools reported providing training to teachers to increase their 
understanding of the specific needs of certain subgroups of students. One of these principals explained, 
“This is a new strategy. Instead of looking at the class as a whole, we really look at each student and what 
[he or she] needs.” As another example, the principal of one school described a new districtwide co- 
teaching model that connected general education teachers with teachers with specialized EL expertise. 
According to this principal, this co-teaching model was intended to result in more aligned instruction 
and improved outcomes for EL students. 


Nearly all of the interviewed principals of priority, focus and other designated low-performing Title | 
schools indicated that building a positive school culture was a central component of their school 
improvement efforts. These principals described various ways in which they were trying to transform 
their school’s culture and climate. Some of the strategies they reported implementing included 
empowering teachers to rebuild their confidence and to meet high expectations for practice, 
implementing formal mentoring and advising programs for students, and launching more informal 
efforts to bring teachers, students, and the broader community together in positive, school-pride- 
building activities. 
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Principals of reward schools described ongoing efforts to foster continuous improvement, including a 
culture of data use and instilling high expectations for all students. Five of the six interviewed 
principals of reward schools described examining data on a continuous basis to promote and sustain 
improvement gains and student achievement successes. These principals described using data not only 
to monitor student progress and provide early interventions to students but also to improve and target 
their schools’ use of available resources. As the principal of one elementary school explained, “It’s just a 
new way of looking at data, and [making] a conscious decision about ways to spend resource money in 
places where we needed help.” 


The six interviewed reward school principals also emphasized that setting and maintaining high 
expectations for students was central to their improvement efforts. These principals described having 
high academic and behavioral expectations for students and promoting a culture in which students take 
ownership, pride, and responsibility for their work in and outside of the classroom. One principal 
described pushing students “to grow and to do all they can do, both academically and from a 
responsibility standpoint.” 


Nearly all states involved in this study (10 of 12) no longer require districts to provide students with 
supplemental education services (SES) or to offer school choice as previously required under NCLB. 
The Department allowed states and districts to choose, under ESEA flexibility, whether they would 
provide SES or public school choice to students attending Title | schools identified for improvement. 
Among the 12 states in the sample, one state continued to require districts to offer SES and school 
choice but reduced the percentage of Title | funds to be set aside for these programs from 20 percent to 
15 percent. In another state, priority schools must use at least 5 percent of their Title | funds to offer 
school choice but are not required to offer SES. Officials in the states that allow districts to choose 
whether to offer these parental-choice options cited a desire to provide districts and schools with 
flexibility over the 20 percent set aside of their Title | funds. All of the interviewed state and district 
officials unanimously supported the flexibility to choose whether to offer SES and school choice. 


Supports and Challenges for Implementing School Improvement Strategies 


ESEA flexibility sought to offer school improvement reform opportunities for states and schools by 
“enabl[ing] states and districts to better advance their work by developing locally tailored solutions to 
address their most pressing education challenges” (U.S. Department of Education, n.d., p. 2). States with 
approval for ESEA flexibility develop plans for implementing systems of differentiated recognition, 
accountability and support based on student achievement in at least reading/English language arts and 
mathematics, graduation rates, and school performance and progress over time, including the 
performance and progress of all subgroups. Specifically, 


States participating in ESEA Flexibility have prioritized their most underperforming schools so 
that they receive additional assistance. ... Compared to NCLB’s mandates for school turnaround, 
state flexibility plans better target strategies on the lowest-performing schools and require 
aggressive action to change a school’s course when necessary. (U.S. Department of Education, 
n.d., p. 2) 
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Officials from the 12 states in this study described requiring or offering support to low-performing 
schools through structures and mechanisms that included external support providers (12 states), early 
warning systems (10 states), and regional organizations (three states). All 12 states in the study sample 
permitted low-performing schools or districts to contract with an external organization or a technical 
assistance provider, but only Massachusetts and Oklahoma mandated such collaboration. Oklahoma 
requires priority schools to partner with local organizations for support in a number of areas, including 
developing family and community engagement strategies. In Massachusetts, districts with priority 
schools are required to identify a Priority Partner from the state’s approved list. Respondents from each 
of the other 10 states in the sample mentioned that their states had contracts with external 
organizations or technical assistance providers but that it was up to districts and/or schools how and 
whether to use these services. 


Three states (Massachusetts, Minnesota and Washington) established state-level processes for 
recruiting, screening, and qualifying vendors to assist schools and districts in selecting high-quality 
external organizations as partners. Officials in three other states (Colorado, New York and Oklahoma) 
reported that they provided guidance to districts and schools to help them choose providers or required 
that any partner vendors demonstrate a track record of success. 


Officials in 10 states were in various stages of developing and implementing early warning systems. The 
interviews with these officials indicated that, despite some variance across states, early warning systems 
had many elements in common. For example, all of these state officials described data management 
systems that collected multiple early warning metrics on students as they progress through the K-12 
system, including measures of student attendance, class grades, assessment results, demographic 
information, and behavior, such as the number of suspensions. State officials reported that their 
systems allow users at the district, school and often classroom level to generate lists of students who 
exhibit certain risk factors, and officials in two states (Massachusetts and Tennessee) reported that their 
systems allow for uploads of additional local data, such as notes from guidance counselors, to account 
for other risk factors that are not captured through the pre-established metrics in the system. 


The officials from three states reported requiring low-performing schools to take advantage of the 
services provided by regional organizations that are staffed by school improvement or turnaround 
experts recruited by the state. These experts support schools in implementing interventions, providing 
professional development to leadership and staff, analyzing data, building school leader capacity, and 
implementing college- and career-ready standards. An official in one of these states said that these 
centers were integral to their plans for school improvement but described challenges finding qualified 
staff for the centers. Officials in the other two states that offered support through regional school 
improvement organizations reported that focus or priority schools were not required to seek support 
from these organizations but that their services were available to schools. 


District leaders in 13 of 21 districts (in nine states) affirmed that they had received support from state 
education agency staff for low-performing schools. District leaders offered mixed reports on the 
quality of the assistance they were receiving from the state. Different states used different names for 
similar roles, but district leaders in five districts said that the state provided “performance managers,” 
“school success coaches,” “school support professionals,” or “turnaround coaches.” These personnel 
met with district and/or school leaders on a regular basis and helped them to identify and implement 
the appropriate interventions, provided professional development, or assisted in other ways. 
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Two district officials lauded the support their districts had received from the state personnel. One 
official specifically noted that the state-assigned performance manager excelled at out-of-the-box 
thinking and brought a fresh perspective on issues their schools were facing. Another official praised the 
state’s use of performance managers to verify that plans were implemented with fidelity, explaining: 


If you’re a district of priority improvement schools, [the state] assigns a performance manager to 
meet with you, at least monthly, to make sure that the plan is actually being implemented. ... 
You do get quite a bit of support from the state, and there is a lot tighter accountability for those 
schools and the districts that are in those categories. 


Some district leaders indicated room for improvement, however, with respect to certain elements of 
their states’ support systems. One district official expressed frustrations and disappointment with the 
school quality auditors who visited district schools on behalf of the state. This official perceived that the 
support was inadequate and ill-timed and that the audits served more as a distraction than a benefit. 
Leaders in another district mentioned that their schools were in need of state staff assistance but that 
none was provided. In addition, with respect to external providers, officials in six districts cited experiences 
with state-approved consultants whom they felt were too far removed from a classroom to provide them 
with high-quality professional development and meaningful assistance around implementation of their 
state’s college- and career-ready standards or the use of data to inform instruction. 


At the school level, all 13 of the priority and focus school principals described receiving some type of 
state support for their improvement strategies. Most of these principals reported receiving state 
support that was related to their improvement strategies, including funding, professional development 
for teachers, or direct support from teams of state staff or intermediary groups funded by the state. For 
example, the principal of a focus school reported receiving an additional $300,000 from the state to 
support the school’s improvement efforts as a result of its focus designation. The principal of a priority 
school in this same state described the support he had received from the state’s educational recovery 
team as central to the school’s improvement efforts and a drastic change in the type of support the 
school had received from the state in the past. This principal stated, “Last year was super. | could not 
have asked for more. | saw [the state] absolutely as a partner in school reform and an advocate for the 
students in the building.” 


The interviewed priority and focus school principals in three states also reported receiving support from 
state-designated school improvement specialists and intermediary groups the state had designated to 
support low-performing schools. These officials indicated that the support was useful, specific to the 
needs of their schools, and either more frequent or more targeted than support offered prior to ESEA 
flexibility. One priority school principal stated: 


They [state staff] would come physically to our building, and if they couldn’t make it [in person], 
they would do a webinar broadcast and say, “This is the scoop.” ... So those were very helpful for 
us, because it did keep that communication stream open. It gave that other venue of information 
that was like, OK, this is straight from the horse’s mouth, sort of thing. ... | thought it was kind of 
nice, since you are on the priority list, your needs were prioritized, so [your needs] got moved to 
the top of the list. 
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Ten of the 13 interviewed priority and focus school principals reported receiving more frequent or 
more intensive support from their districts under ESEA flexibility than in previous years—specifically 
with regard to data use. Ten of the 13 interviewed principals specifically reported that they had 
received support from their district in reviewing and interpreting data and that the support the district 
had provided was useful, relevant and available to them as needed. As the principal of one priority 
school remarked, “Hardly a week goes by that some sort of data analysis training doesn’t take place.” 
Similarly, other school principals reported that their districts regularly monitored their data and 
described their district staff as “on call” to answer questions, provide data reports and support the 
school in interpreting data reports. In addition to data analysis support, principals of priority and focus 
schools across states in the study sample reported that their districts had provided them with site-based 
reading and mathematics specialists or coaches. Principals also highlighted the support of district-based 
instructional specialists who were visiting their schools regularly to lead professional learning 
community meetings, attend school leadership team meetings, and assist the school in implementing 
the activities and strategies in the school improvement plan. 


Principals from 20 of the 25 schools in the sample described a variety of factors that challenged their 
improvement efforts, including low staff morale (10 principals), limited resources (nine principals), 
staff turnover (three principals), and testing requirement burdens for EL students (two principals). 
Among the nine principals citing limited resources as a challenge, three were priority school principals, 
five were reward school principals, and one was a focus school principal. The priority and focus school 
principals described challenges related to allocating limited resources to address the multiple needs of 
the school or the prospect of losing SIG funding, which supported many of their improvement efforts. 
The five reward school principals all cited funding as a challenge. These principals indicated that because 
of their reward school status, their schools typically were not targeted to the same extent as priority and 
focus schools to receive additional funds or other resources from the state or district. 


Burden Associated With ESEA Flexibility and State and District Capacity to 
Support the Implementation of ESEA Flexibility Provisions 


The full set of activities involved with ESEA flexibility—developing the policies, soliciting feedback, 
revising requests, putting policies in place, and providing technical support—involved a great deal of 
effort on the part of education officials from all levels. In addition, the scope and extent to which state 
accountability systems were changed through ESEA flexibility provisions placed significant demands on 
state and district capacity to identify high- and low-performing schools successfully and to provide 
effective support to struggling schools. The study team asked respondents to gauge the extent of 
burden they experienced in transitioning to the new accountability systems as well as their capacity to 
effectively implement their systems. 


The interviewed state and district officials reported that ESEA flexibility implementation required 
substantial effort; most state officials in the sample described at least a modest level of burden, but 
the majority of district officials reported that the level of effort was not burdensome. Among the 12 
state respondents, eight described at least a modest burden associated with ESEA flexibility 
implementation, and another three respondents described a substantial effort but not necessarily a 
burden. Among the eight state officials who reported at least a modest burden, one state administrator 
explained: 
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There seems to be a misunderstanding at the [U.S.] Department [of Education] about the 
amount of time that it takes for us to pull together the information that’s requested. ... In order 
to change completely our accountability system, we had to rewrite every single program and 
communicate with the districts at the same time. ... So then, when we would get a call from the 
Department saying, “Well, could you just show us the impact if you were to change this 
percentage?” That seems to them like a very minor request, but for us it was not, in the midst of 
everything else that we were trying to do. 


Another state-level interviewee described “a significant amount of workload” associated with 
responding to inquiries about ESEA flexibility from stakeholders. 


Officials in three other states noted that although implementing the ESEA flexibility provisions for 
differentiated accountability and support systems entailed a substantial effort, they did not perceive the 
level of effort as burdensome. For example, one official explained: 


Any time you make a change, there is the obligation to make sure that our districts are informed 
and our internal staff are informed. So obviously that took a little bit of time in getting everyone 
to acommon understanding of what was required under the ESEA Flexibility waiver. | wouldn’t 
necessarily consider that burdensome because it was an improvement to the entire system. 


At the district level, officials in 19 of the 22 districts in the sample provided data on level of 
implementation burden associated with the accountability systems under ESEA flexibility. Among these 
19, seven described implementation to be burdensome, but officials from 12 districts reported effort but 
no burden associated with implementation. One district official reported that “any change in approach 
takes learning in general” but that it has not been particularly burdensome. This sentiment was common 
among the other 12 respondents who reported effort but no burden associated with implementation. 


State officials described implementing strategies to build capacity to implement the accountability 
and support provisions under ESEA flexibility. The interviewed state officials described a variety of 
state-level efforts to enhance the state’s capacity, including cross-agency collaborations, developing 
staff expertise, and partnerships with districts (seven states); structural changes within the state 
educational agency to enhance the state’s capacity (eight states); and efforts to better align and 
coordinate oversight of federal initiatives and/or multiple funding streams (five states). For example, the 
officials who described cross-agency collaborations explained that the shared efforts and responsibilities 
across agency staff and direct links to support low-performing schools bolstered their capacity to 
facilitate districts and schools in their improvement efforts. Among the officials who reported structural 
changes and efforts to coordinate federal initiatives and funding streams, these respondents primarily 
described shifting roles and responsibilities within the state educational agency to create staff and units 
that would have joint responsibility for accountability and supports. 
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A majority of district officials (15 of 22) reported that they have sufficient staff and expertise to 
support continuous improvement in all priority, focus and other designated low-performing Title | 
schools. Many district officials reported developing and using both internal and external expertise to 
improve school turnaround efforts. For example, one district official reported that focusing on meeting 
ESEA flexibility requirements encouraged the district to tap “into expertise from local colleges, 
universities, as well as national consultants.” Several district officials reported strategies for enhancing 
their capacity. Three district officials reported that they were in the process of hiring additional staff 
members to support low-performing schools. Another three reported that they would like to hire more 
administrative staff but were unable to do so because of funding constraints. Four districts reported that 
they planned to work with external providers to support their low-performing schools. 


Despite efforts to build capacity, nearly all of the state officials and half of the district officials 
described challenges or concerns related to their existing capacity to support improvement in low- 
performing schools. Although state officials reported that the smaller caseloads of schools helped them 
better meet these schools’ needs, officials from five states expressed concern that the state still had too 
few staff to provide the level of individualized supports to schools that is necessary to promote 
sustainable school turnaround. Officials from two states also expressed concern about the state’s 
capacity to support schools that exit the priority and focus designations to the extent that is needed to 
sustain those schools’ improvement in the long term. 


The majority of district officials (15 of 22) reported that they presently had sufficient staff and expertise 
to support school improvement in identified schools; however, officials from seven districts reported 
factors constraining their capacity, including insufficient funds (four districts), a lack of expertise in 
intervening in low-performing schools (two districts), and the complex demands of school improvement 
(one district). Officials from an additional four districts expressed concern about their ability to sustain 
their capacity to support schools at scale after their SIG and/or Race to the Top grants expire. 
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Conclusion 


By extending flexibility on certain provisions of ESEA, as amended, the Department aims to attenuate 
many of the provisions that were perceived as hindering reform and to encourage innovative efforts to 
identify and support low-performing schools. States with approved ESEA flexibility requests are granted 
flexibility with regard to annual measurable objectives, AYP determinations, identification of and 
supports and interventions for low-performing schools and districts, the use of section 1003(a) funds, 
the transferability of ESEA funds, and improvement plans for highly qualified teachers. In order to be 
approved to implement ESEA flexibility, these states must adhere to specific principles: (1) ensure 
college- and career-ready expectations for all students; (2) implement state-developed differentiated 
recognition, accountability, and support; (3) support effective instruction and leadership; and (4) reduce 
duplication and unnecessary burden. 


The purpose of this study was to explore the early implementation of state systems of differentiated 
support and accountability under ESEA flexibility. The study sample included 25 schools in 22 districts 
located within 12 of the 35 states that were granted flexibility in Window 1 and Window 2. The key 
findings from the study follow. 


For most states in this study, their accountability systems under ESEA flexibility build on prior state 
policies and reform efforts. At the same time, state officials indicated that the flexibility provisions 
allowed them to address perceived deficiencies of NCLB. For example, state officials reported 
developing new state accountability systems under ESEA flexibility that built off of the NCLB growth 
model pilot, differentiated accountability, Race to the Top, and state-specific reforms. Although states 
perceived ESEA flexibility as an opportunity to continue these types of reform efforts already in place, 
state officials also indicated that flexibility enabled them to address challenges they had encountered 
with NCLB, such as the over-identification of low-performing schools and the over-reliance on 
proficiency in reading and mathematics as school performance metrics. Likewise, principals appreciated 
accountability systems that took into account performance on measures other than proficiency in 
reading and mathematics, with increased focus on student academic growth and reduction of 
achievement gaps among subgroups. 


Many states that sought to reconcile their “dual” state and federal systems of accountability through 
ESEA flexibility described continued challenges in this regard. Although some state officials reported 
that one of their goals in applying for ESEA flexibility had been to reconcile their state accountability 
system and the federal accountability system, this goal was only partially realized in these states. Thus, 
ESEA flexibility helped better align state and federal systems of accountability, but some officials 
reported that it had not eliminated a “dual” system of accountability. 
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Interviews with principals revealed differences across states in school principals’ understanding of 
state accountability systems. Principals in four of the six states from which the school sample was 
selected reported that their states and districts had done a good job of explaining the state system of 
accountability and support under ESEA flexibility. They reported that the communication efforts of their 
states had provided them with a clear understanding of the system, including the implications of their 
school designations. In comparison, the principals in two other states described the opposite: that their 
states had provided them with insufficient information to support their full understanding of the system 
and the process by which their schools were designated. Notably, these principals also expressed concern 
about the validity of the criteria their state was using to make school accountability designations. 


Targeted support is another critical component of ESEA flexibility, and district leaders in 13 of 21 
districts (in nine states) affirmed that they had received support from the state educational agency. 
However, district leaders offered mixed impressions of the assistance they were receiving from the 
state. Officials from four districts reported increased state support to develop and implement 
interventions for low-performing schools. For example, one district administrator lauded the support his 
district had received, mentioning that the assigned performance manager excelled at out-of-the-box 
thinking and brought a fresh perspective on issues their schools were facing. In contrast, a leader of a 
district in a different state expressed frustration with the school quality auditors assigned by the state, 
who were described as a distraction rather than a benefit. 


All of the priority and focus school principals in the sample described receiving state support for their 
improvement strategies as a result of their designation status. State officials reported that the smaller 
numbers of schools identified as low performing allowed them to provide better support and conduct 
more substantive progress monitoring. Moreover, 10 of the 13 interviewed priority and focus school 
principals reported receiving more frequent or more intensive support from their districts under ESEA 
flexibility than in previous years—specifically with regard to data use. Indeed, district officials reported 
that smaller caseloads allowed school improvement specialists and academic coaches to visit schools 
more often for progress monitoring and professional development. 


Officials from seven districts reported insufficient staff and expertise to provide the level of support 
they believed was needed to improve low-performing schools in their districts. As reported by officials 
in these seven districts, the factors that constrained capacity included insufficient funds (four districts), a 
lack of expertise in intervening in low-performing schools (two districts), and the complex demands of 
school improvement (one district). 


Among the six reward schools in the sample, only one school principal reported receiving formal 
recognition from the state associated with the school’s reward status. None of the reward school 
principals reported receiving incentives in the form of monetary rewards or compensation as a result of 
their reward school designation. 
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Appendix Exhibits 


Exhibit A1. School-Level Characteristics of Participating and Non-Participating Schools in the Study 


Sample 
School-Level Characteristics Participating Schools Non-Participating Schools 
Average enrollment 550 565 
Average percentage of students receiving free or 74% 76% 
reduced-price lunch 
Average percentage of minority students 72% 79% 
Average percentage of special education students 14% 19% 
Sample size N=25 N=11 


Source: 2011-12 EDFacts, Data Group (DG) 39: Membership, DG 74: Children with disabilities (IDEA) school age, DG 565: Free 
or reduced-price lunch. 
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Exhibit A2._ Non-Financial Rewards Offered to Reward Schools 
State Types of Public Recognition Regulatory Flexibility 
Autonomy Exemptions From 
Over Funds Regulations 
Colorado Certificate as Title | Distinguished School 
Press release 
Media event 
Kentucky Permission to schools to use recognition status on 
materials 
Maryland Press release Exempt from 
Display on state website and other promotions completing most 
Certificate of recognition from state and district parts of annual 
Distinguished schools receive plaque school 
Recognition of superlative schools from governor and improvement plan 
state board of education, and visit from dignitaries 
Massachusetts State-level event 
Certificates given to staff 
Minnesota Recognition in annual press conference with governor 
and education commissioner 
Certificates given to staff 
Missouri Letter of congratulations 
Award from state board of education 
New Jersey Schools identified as New Jersey Schools of Excellence 
Plaques for school and district 
Statewide press releases 
Visit from governor and/or commissioner of 
education 
Selected schools or students asked to present at 
Effective Practices Conference 
New Mexico Press release Exempt from 
Showcase on website/online clearinghouse completing most 
Letter of recognition from governor and education parts of annual 
secretary school 
improvement plan 
New York Press release Specifics under 
Showcase on website discussion 
Eligible for Commissioner’s Schools Dissemination 
Grant 
Oklahoma Honor at state board of education meeting Increased 
Press release autonomy 
Representation on advisory groups over federal 
Inclusion in state superintendent’s listening tour funds 
Tennessee Various forms of public recognition 
Washington State visit and award ceremony at school 


State dissemination of materials lauding school 


Source: State department of education websites, state ESEA flexibility approved requests, and interviews with state officials. 
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Exhibit A3. Identification of Other Low-Performing Schools 
State School Category Definition 
Colorado Turnaround and Schools identified as low performing using the state’s school and 
Priority Improvement district performance frameworks that have not also been identified 

as priority or focus schools 

Maryland Approaching Target Title | schools that do not meet the criteria for priority or focus 
schools under ESEA flexibility but that do not/did not meet AMOs in 
one or more categories 

Massachusetts Level 3 (not focus) Schools with low aggregate performance that are among the 
lowest 20 percent relative to other schools in their grade span 
statewide and that are not already identified as priority or focus 
schools 

Minnesota Continuous The bottom 25 percent of Title | schools that have not already been 

Improvement identified as priority or focus schools 

New Mexico Strategic Title | schools with the largest achievement gaps in the state and 
with an overall school grade of “C” or lower, as measured by the 
state A-F grading system 

New York Local Assistance Any school that is not a priority or focus school that (a) has large 
gaps in student achievement among subgroups of students, (b) has 
failed to make AYP for three consecutive years with same subgroup 
on same measure, or (c) is located in a non-focus district but is 
among the lowest in the state for the performance of one or more 
subgroups and for which the school is not showing progress 

Oklahoma Targeted Intervention For 2012-13, any schools in the bottom 25 percent of the state 
(based on student achievement) that have not previously been 
identified as priority or focus schools 

Washington Emerging Title |-participating schools identified from the priority list (next 5 


percent up from the bottom of the list) and the focus list (next 10 
percent up from the bottom) that are not already identified as 
priority or focus schools 


Source: State department of education websites, state ESEA flexibility approved requests and interviews with state officials. 


Note: Among the 12-state sample, four states (Kentucky, Missouri, New Jersey and Tennessee) did not have a specific school 
accountability designation for identified low-performing schools that did not meet the criteria for designation as a priority or 


focus school. 
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